Semantic Segmentation with Same Topic Constraints

A popular approach to semantic segmentation problems is to construct a pair wise Conditional Markov Random Field (CRF) over image pixels where the pair wise term encodes a preference for smoothness within pixel neighborhoods. Recently, researchers have considered higher-order models that encode local region or soft non-local constraints (e.g., label consistency or co-occurrence statistics). These new models with higher-order terms have significantly pushed the state-of-the-art for semantic segmentation problems. In this study, we consider a novel nonlocal constraint that enforces consistent pixel labels among those image regions having the same topic. These topics are discovered by Probabilistic Latent Semantic Analysis model (PLSA). We encode this constraint as a robust Pn higher-order potential among all the image regions of the same topic in a unified CRF model. We experimentally demonstrate quantitative and qualitative improvements over a refined baseline unary and pair wise CRF models.